{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "16817097",
   "metadata": {},
   "source": [
    "# Getting started with development in `sktime` on Windows with PyCharm and Anaconda \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57b16167",
   "metadata": {},
   "source": [
    "## Objectives:\n",
    "Set up and install a development environment for `sktime` in Windows. \n",
    "This will involve:\n",
    "* cloning the repository within the **PyCharm** IDE **[(Section 1)](#Section-1:-Cloning-sktime-in-PyCharm)**\n",
    "* setting up an **Anaconda** environment for installing packages and running code **[(Section 2)](#Section-2:-Setting-up-an-Anaconda-environment-and-installing-sktime)**\n",
    "* configuring PyCharm to use the Anaconda environment and demonstrate how to run local tests with `pytest` **[(Section 3)](#Section-3:-Configuring-PyCharm-and-running-local-tests-with-pytest)**\n",
    "* demonstrating the use of `pre-commit` to reject and tidy messy code before committing **[(Section 4)](#Section-4:-Configuring-pre-commit-with-PyCharm)**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec8b5b5d",
   "metadata": {},
   "source": [
    "> **_NOTE:_**  This guide assumes you are starting from first principles with no existing environment/IDE configuration settings. The IDE that is used is PyCharm - this IDE is not a requirement for `sktime` development and you are free to use any/no IDE. **If you would like to only create an Anaconda environment and install `sktime` with its dependencies, please make sure you have cloned `sktime` through other means and skip down to Section 2**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "364c76e9",
   "metadata": {},
   "source": [
    "## Pre-requisites: \n",
    "* [Anaconda](https://www.anaconda.com/products/individual#windows) (the version available at the time of writing was Anaconda3-2021.11) Anaconda will be used for installing `sktime` with its dependencies into a virtual environment.\n",
    "* [PyCharm](https://www.jetbrains.com/pycharm/download/#section=windows) (the version available at the time of writing was 2021.2.3) PyCharm is the IDE that will be used in the following example, but developers are not restricted to using PyCharm if they would prefer to use another IDE.\n",
    "* [Build Tools for Visual Studio 2022/Visual Studio 2022 with Desktop Development in C++](https://visualstudio.microsoft.com/downloads/?q=build+tools) A C++ compiler is required for compiling Cython files during the full installation of the development version of `sktime`. This can be achieved by just installing the Build Tools for Visual Studio or, if you prefer, you can install the full Visual Studio editor (with either installation, make sure to tick the **Desktop development in C++** option when installing to ensure that a C++ compiler is included).\n",
    "* (Optional) Git for Windows - This is required, and you can install it manually if you wish, but PyCharm can do this for you and it is included in the instructions for Step 1 below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0374c765",
   "metadata": {},
   "source": [
    "# Section 1: Cloning `sktime` in PyCharm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66f1bafa",
   "metadata": {},
   "source": [
    "### 1. Open **PyCharm** and select the **Get from VCS** option:\n",
    "![03_pycharm_start.png](img/windows_installation/03_pycharm_start.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fc6c52a",
   "metadata": {},
   "source": [
    "### 2. You should see the following screen: \n",
    "\n",
    "![04_pycharm_no_github_no_git.png](img/windows_installation/04_pycharm_no_github_no_git.png)\n",
    "\n",
    "The example above shows an installation where **the user is not logged into GitHub** within PyCharm and **does not have Git (Git for Windows) installed**. Step 3 demonstrates how to log in to GitHub (useful for later) and Step 4 demonstrates how to install Git. If you have already completed either of these steps for another project, please skip ahead to Step 5 where the project is cloned. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "871827ae",
   "metadata": {},
   "source": [
    "### 3. To log into your GitHub account, click the GitHub tab and then the *Log In via GitHub...* option. \n",
    "\n",
    "![05_login_with_github.png](img/windows_installation/05_login_with_github.png)\n",
    "\n",
    "This should launch your browser where you can log in to your account. Once you complete this step, you can return to PyCharm and you should see that your GitHub handle is now listed where it previously stated *No accounts*. This is where you could clone any of your own repositories if you wish, but clock back to the *Repository URL* option to now install Git and clone `sktime`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72ace5b5",
   "metadata": {},
   "source": [
    "### 4. If Git is not installed on your machine then you should see the following highlighted message:\n",
    "\n",
    "![06_install_git_within_pycharm.png](img/windows_installation/06_install_git_within_pycharm.png)\n",
    "\n",
    "Click the *Download and Install* link and PyCharm will automatically install Git for Windows on your machine."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2592811a",
   "metadata": {},
   "source": [
    "### 5. If you are correctly logged into your GitHub account with Git installed then you should now see your username on the left under the **GitHub** tab and you should see the message **Git has been installed** underneath the two text fields:\n",
    "\n",
    "![07_git_installed_enter_sktime_url.png](img/windows_installation/07_git_installed_enter_sktime_url.png)\n",
    "\n",
    "With your GitHub account logged in, an Git installed, it is now time to clone `sktime`. Enter the URL of the GitHub repository (https://github.com/sktime/sktime) into the **URL** field, set the location where you would like the files to be downloaded to in the **Directory** field, and press the **Clone** button at the bottom of the prompt to initiate the download of `sktime` to your local machine."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76da2044",
   "metadata": {},
   "source": [
    "### 6. Once PyCharm has cloned the project (this can take a minute or so) you should see the following screen:\n",
    "\n",
    "![08_sktime_in_pycharm_no_interpreter.png](img/windows_installation/08_sktime_in_pycharm_no_interpreter.png)\n",
    "\n",
    "In particular, the bottom-right corner indicates that the IDE is on the `main` branch of `sktime` but no interpreter is selected for this project (if you have used PyCharm before then it may default to an existing interpreter from another project, however). We will now switch to Anaconda to set up an environment and install the necessary libraries to develop with `sktime`. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b178c7b0",
   "metadata": {},
   "source": [
    "# Section 2: Setting up an Anaconda environment and installing `sktime`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed910245",
   "metadata": {},
   "source": [
    "> **_NOTE:_**  These instructions are valid as of 24th November 2021. There are known issues with the `fbprohpet` dependency and the instructions below include a temporary work-around. This will be addressed in a future update and these instructions will be updated to reflect this"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "034b1064",
   "metadata": {},
   "source": [
    "> **_NOTE:_**  If you are already familiar with Anaconda/this is not your first time following these instructions and would like a shortcut then the steps below walk-through these specific instructions: \n",
    "\n",
    "\n",
    "```\n",
    "    conda create --name sktime python=3.8\n",
    "    conda activate sktime\n",
    "    pip install -e .[all_extras]\n",
    "    pip install pytest\n",
    "    pip install esig\n",
    "    conda install -c conda-forge fbprophet\n",
    "    pip install scipy --force-reinstall\n",
    "    pip install numpy==1.19.3 --force-reinstall\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fdbfbfa",
   "metadata": {},
   "source": [
    "### 7. Create a new Anaconda environment for `sktime` development\n",
    "Open Anaconda prompt to create a new conda environment for `sktime` development. We will also call the environment `sktime` in this example (you can use a different name if you wish as long as you are consistent when initialising the env and also selecting it later in PyCharm)\n",
    "\n",
    "`sktime` currently supports Python version 3.8, hence we will create the new environment with this requirement using the following statement: \n",
    "\n",
    "```\n",
    "conda create --name sktime python=3.8\n",
    "```\n",
    "You should see the following where you will be  prompted to accept/reject the proposed package plan - accept by typing `y` and pressing enter\n",
    "\n",
    "![09_create_conda_env.png](img/windows_installation/09_create_conda_env.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bdd5c5a",
   "metadata": {},
   "source": [
    "### 8. Activate the environment and install `sktime`\n",
    "Once the environment is created you can activate it with `conda activate <env_name>` where `env_name` is the name of the environment that you have just created. In this case, we will use: \n",
    "```\n",
    "conda activate sktime\n",
    "```\n",
    "Once your new environment is correctly activated you should see that `(base)` changes to the name of your environment (`sktime` in this example). \n",
    "\n",
    "We are now ready to navigate to the **root directory** of your `sktime` project to `pip install` the setup file for `sktime`. We will do this with the command:\n",
    "```\n",
    "pip install -e .[all_extras]\n",
    "```\n",
    "where `-e` means that it is installed in editable mode (useful for development), `.` specifies that the installation instructions should come from the current directory, and `[all_extras]` means that all soft dependencies will be included. This also means that Cython will be included in the installation and any Cython code in the repository will be compiled on your  machine during this command's execution (this step can therefore take a minute or two to complete). A screenshot of what it should look like when activating your environment and navigating to your `sktime` root directory is as follows:\n",
    "\n",
    "![10_activate_navigate_install.png](img/windows_installation/10_activate_navigate_install.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f28b1b1",
   "metadata": {},
   "source": [
    "> **_NOTE:_**  As mentioned at the beginning of Section 2 there is a known issue with `fbprophet` when being installed this way. It is expected that you will see red text and errors referring to `fbprophet` here and we will address this in the subsequent steps. You can expect to see error output such as the following, but note that below the error, the other packages are being correctly installed at this stage: \n",
    "![11_expected_fbprophet_output.png](img/windows_installation/11_expected_fbprophet_output.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be05acbd",
   "metadata": {},
   "source": [
    "### 9. Install `pytest` and `esig`\n",
    "These two packages are not currently included as dependencies but are useful for development. Installation of both is straight forward with:\n",
    "```\n",
    "pip install pytest\n",
    "pip install esig\n",
    "```\n",
    "Successful installation of both packages should appear as follows:\n",
    "\n",
    "![12_install_pytest_esig.png](img/windows_installation/12_install_pytest_esig.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1b8a385",
   "metadata": {},
   "source": [
    "### 10. Install `fbprophet` via `conda` rather than `pip`\n",
    "The next step is to rectify the previous issues with installing `fbprophet`. To do this, call:\n",
    "```\n",
    "conda install -c conda-forge fbprophet\n",
    "```\n",
    "\n",
    "This will bring a large number of packages and updates and you should see a screen similar to the following: \n",
    "\n",
    "![13_conda_install_fbprohpet.png](img/windows_installation/13_conda_install_fbprohpet.png)\n",
    "\n",
    "Accept the installation and we are *nearly* done..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2fcdb39",
   "metadata": {},
   "source": [
    "### 11. Reinstall `scipy` and `numpy`\n",
    "\n",
    "Installing `fbprophet` in this way modifies two previously installed dependencies. `scipy` must be reinstalled and, when doing so, this upgrades `numpy` to a version higher than other packages can support. Therefore we must run two further commands to first reinstall `scipy`, and then another to downgrade `numpy` again. These commands are as follows:\n",
    "```\n",
    "pip install scipy --force-reinstall\n",
    "pip install numpy==1.19.3 --force-reinstall\n",
    "```\n",
    "Once complete, your terminal should appear similar to the following:\n",
    "\n",
    "![14_reinstall_cipy_downgrade_numpy.png](img/windows_installation/14_reinstall_cipy_downgrade_numpy.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cdfa60a",
   "metadata": {},
   "source": [
    "### Congratulations - `sktime` is now installed and your environment is ready to use!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb8aa6ad",
   "metadata": {},
   "source": [
    "# Section 3: Configuring PyCharm and running local tests with `pytest`\n",
    "At the end of Section 1 we saw that our PyCharm project did not have an interpreter (or you may have a default environment/an existing environment that has been detected for another project). We now want to set our PyCharm project to use the newly created Anaconda environment from Section 2.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fa02b6f",
   "metadata": {},
   "source": [
    "### 12. Open File -> Settings\n",
    "\n",
    "![15_pycharm_interpreter_settings.png](img/windows_installation/15_pycharm_interpreter_settings.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11a3040f",
   "metadata": {},
   "source": [
    "### 13. Select *Python Interpreter* from the *Project* heading. Click the gear icon at the top-right and select *Add...* when the option appears\n",
    "\n",
    "![16_add_new_env.png](img/windows_installation/16_add_new_env.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a180fa93",
   "metadata": {},
   "source": [
    "### 14. Add our existing environment to this project\n",
    "We will now add our existing Anaconda environment as this project's interpreter. Select *Existing interpreter* and then click the `...` button to navigate for the `python.exe` for your Anaconda environment (note: this should be found wherever you installed Anaconda within the `env/<your_env_name>` directory). For example:\n",
    "\n",
    "![17_add_new_conda_env.png](img/windows_installation/17_add_new_conda_env.png)\n",
    "\n",
    "Press *OK* to confirm and you can now close the project settings. Your environment is now ready to use with your PyCharm project, so let's test this by running a test locally with `pytest`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5af9b5c",
   "metadata": {},
   "source": [
    "### 15. Run `test_all_panel_estimators`\n",
    "\n",
    "This step will demonstrate how we can run tests locally in PyCharm. The objective of this is to both demonstrate how to run tests, but also to verify your installation of `sktime`. We will run the `test_all_panel_estimators` file as these tests require the dependencies to be correctly installed and also for the Cython code within `sktime` to be correctly compiled on your machine. To run these tests, simply expand `sktime/series_as_features/tests` and right-click `test_all_panel_estimators` and select *Run 'pytest in test_all_p...'* as follows:\n",
    "\n",
    "![18_run_tests_to_verify.png](img/windows_installation/18_run_tests_to_verify.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b627ed4",
   "metadata": {},
   "source": [
    "### 16. Verify all tests passed\n",
    "\n",
    "At the time of writing `test_all_panel_estimators` contained 117 tests that should all pass if the IDE and environment are configured and installed correctly (this number may change in future releases if tests are added/removed). \n",
    "\n",
    "If you have reached this step then you should be ready to write your own contributions for `sktime`. Once you have done that you will want to make sure that you have correctly configured `pre-commit` before trying to push any code, however...\n",
    "\n",
    "![19_test_success.png](img/windows_installation/19_test_success.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de81066c",
   "metadata": {},
   "source": [
    "# Section 4: Configuring `pre-commit` with PyCharm\n",
    "\n",
    "`pre-commit` is a very useful package for checking your code for simple style errors at the commit stage. This is very useful when working on large collaborative projects as it allows code reviewers to focus on the function of new code rather than conformity to style. For example, consider the following code: \n",
    "\n",
    "![21_pre-commit_examples.png](img/windows_installation/21_pre-commit_examples.png)\n",
    "\n",
    "The code is not very useful but it is clear that it contains a number of formatting errors: \n",
    "1. line 3 includes unnecessary spaces\n",
    "2. line 3 includes an unnecessary semi-colon at the end of the line\n",
    "3. there are multiple blank lines at the end of the file (lines 10 to 14)\n",
    "4. there is an unused import on line 1\n",
    "5. the variable on line 6 does not conform to Python naming conventions \n",
    "\n",
    "We will see that `pre-commit` can help with *some* of these issues (e.g. 1-3) but it is not a magic bullet that can fix everything - the emphasis is still on the programmer to make sure that their code conforms to required conventions and `pre-commit` can help with this."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b946a36a",
   "metadata": {},
   "source": [
    "### 17. Install  `pre-commit`\n",
    "Switch back to Anaconda prompt (or open a new window and activate your development environment if you are returning) and run:\n",
    "```\n",
    "pip install pre-commit\n",
    "```\n",
    "and then\n",
    "```\n",
    "pre-commit install\n",
    "```\n",
    "\n",
    "You should see the following:\n",
    "\n",
    "![20_install_pre-commit.png](img/windows_installation/20_install_pre-commit.png)\n",
    "\n",
    "> **_NOTE:_** When running `pre-commit install`, if you see a message stating that Git is not installed then it is likely that you need to update your system Path variables to include Git. To do this, press the Windows key and search for *Edit the system environment variables*. On the **System Properties** dialog that appears, select the *Advanced* tab and press the *Environment Variables...* button on the bottom-right. Double-click the entry for **Path** and add `C:\\ProgramFilesGit\\bin\\` and `C:\\ProgramFilesGit\\cmd\\` (these paths may be slightly different on your installation). Once you have updated the variables, press *OK* on all of the windows to close them and restart Anaconda Prompt - if this was your issue, `pre-commit install` should now work once you activate your environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c1d9846",
   "metadata": {},
   "source": [
    "### 18. Attempt to commit some code\n",
    "In the example below we are trying to commit the poorly formatted code from above. To do this, click *Git -> Commit...* and this will open the *Commit* pane (this may appear in a separate window if your IDE is configured differently)\n",
    "\n",
    "![22_create_new_commit.png](img/windows_installation/22_create_new_commit.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3f73f13",
   "metadata": {},
   "source": [
    "### 19. Ensure that *Run Git hooks* is ticked\n",
    "\n",
    "You can find this option by selecting the gear icon and it is the last option in the *Before Commit* section:\n",
    "\n",
    "![23_ensure_check_hooks_ticket.png](img/windows_installation/23_ensure_check_hooks_ticket.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "020223c4",
   "metadata": {},
   "source": [
    "### 20. Attempt to commit the poorly formatted code\n",
    "\n",
    "Ensure that a commit message is included and then attempt to commit the code. If your code has similar issues to the example above then the commit will fail. This is indicated by the pop-up in the bottom-right corner and selecting *Show details in console* will explain what exactly failed. A convenient feature of `pre-commit` is that the issues are automatically fixed in the source code too: \n",
    "\n",
    "![24_failed_commit_with_errors_and_automatic_correction.png](img/windows_installation/24_failed_commit_with_errors_and_automatic_correction.png)\n",
    "\n",
    "**However**, it is important to note that `pre-commit` did not remove the unused import or correct the variable name. Therefore, it is very useful for helping you to produce tidy code but it is not a guarantee, and it is important to still manually check that your code conforms to `sktime` style guidelines. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "869dd3fc",
   "metadata": {},
   "source": [
    "# The End. \n",
    "\n",
    "Congratulations, you should now have a correctly configured development environment for `sktime` using PyCharm, Anaconda, `pytest` and `pre-commit`. If you would like to see examples in code before getting started then please make sure to have a look at the JuPyter notebooks that are included in the *examples* directory and in the `sktime` documentation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
